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DETAILED ACTION 
Response to Amendment 

Applicant's submission filed on 6/11/2007 has been entered. Claims 1, 6, 11 and 16 have 
been amended. The claims 1-25 are pending in the application. 



Response to Arguments 

Applicant's arguments filed June 11, 2007 have been fully considered but are moot in 
view of the new ground(s) of rejection of the base claim 1 based on Kahn et al. US 
2004/0004663 Al and Asami U.S. Patent No. 6,882,350 set forth in the present Office Action. 

Kahn teaches a method for generating an album based on album data including at least 
one image data set, which has been photographed during a trip and which has location data 
representing a photography location attached thereto, comprising the steps of: 

Loading the image data into a personal computer (See Paragraph 0044-0045 wherein the 
image data is taken by the digital camera 100 and is loaded to the local host device such as cell 
phone, personal digital assistant, computer or the like and the personal digital assistant or the 
computer meets the claim limitation of "a personal computer"); 

Forwarding image data from a user terminal of said personal computer to a generation 
support server (e.g., Kahn teaches forwarding/uploading user's photos from the local host device 
to the image management server of Fig. 4; see Paragraph 0089 and 0097-0098), a user at the 
user terminal being identified by a user ID (user 's GPS location, location ID in paragraph 0094 
and 0097 and user 's identity ID of paragraph 0098), where a predetermined reference position 
corresponds to said user ID (e.g., Paragraph 0051 wherein the location information of where the 
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user is currently located, the most recently captured images for organizing a plurality of photos. 
The location information may also be a centroid of a given geographic location of interest of 
Kahn meets the claim limitation of "a predetermined reference position " wherein the centroid is 
associated with the user's location ID and/or the user's ID; see paragraph 0093); and, 
The generation support server performing the steps of: 

Calculating said distance between the photography location of the image data set and a 
predetermined reference position, based on the location data {e.g., Kahn teaches calculating the 
distance by determining whether the user 's GPS location lies within a given geographic location 
of interest to determine the distance between the user f s GPS location and the centroid for a 
given geographic location as performed within an SQL stored procedure; see Paragraph 0093); 

Judging whether the distance is over a predetermined threshold value (e.g., the radius of 
a given geographic location as taught in Kahn meets the claim limitation of "a predetermined 
threshold value"; see Paragraph 0093-0094; a list of fulfillers within a given distance as 
retrieved from the database in the image manager are returned to the local applet and displayed 
to the user as sorted by least distance to the user; see Paragraph 0098-0100); 

Classifying the image data set according to the result of judgment ("albums" based on 
location of Paragraph 0098-0100); and 

Generating the album data according to the result of classification ( "albums " based on 
location of Paragraph 0098-0100). 

Kahn is silent to the claim limitation of displaying together, on a display screen, both 
map data indicating a travel route and thumbnail images in a chronological order. 
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However, Asami discloses in Figs. 10-12 and 15-19 displaying together on a display 
screen both map data indicating a travel route and thumbnail images in a chronological order 
(See also Asami column 20, lines 34-62, e.g., the map data indicating the sightseeing course of 
the tour or the map data indicating the route connecting the thumbnail icons of photos). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Kahn's method and apparatus for generating an 
album because Kahn's photos are location dependent in a trip and are ordered in chronological 
order in a trip and Kahn's photos thus can be displayed according to the teaching of Asami (Figs. 
15-19 and 10-12 of Asami) as Asami's photos are also locations dependent and are ordered 
chronologically for a trip. 

One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19). 

Applicant's arguments filed June 11, 2007 have been fully considered but are moot in 
view of the new ground(s) of rejection of the base claim 1 also based on Loui et al. U.S. Patent 
No. 6,636,648 (hereinafter Loui), in view of Wilcock et al. U.S. Patent No. 6,741,864 
(hereinafter Wilcock) and Asami U.S. Patent No. 6,882,350 (hereinafter Asami). 

Re claims 1, 6, 11, 16 and 25, Loui discloses a method for generating an album based on 
album data including at least one image data set, which has been photographed using a digital 
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camera during a trip and which has location data representing a photography location attached 
thereto, comprising the steps of: 

Calculating a distance between the photography location of the image data set and a 
predetermined reference position, based on the location data {e.g., Loui teaches in column 9-11 
adding annotations for pictures and associating metadata to the pictures. Loui teaches an 
albuming algorithm that arranges the picture according to an event or a sub-event wherein the 
event or sub-event are defined by the metadata including metadata tags and GPS information to 
determine location and subject depth information. The information defines the context 
surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries); 

Judging whether the distance is over a predetermined threshold value (e.g., Loui teaches 
in column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 
teaches an albuming algorithm that arranges the picture according to an event or a sub-event 
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wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries); 

Classifying the image data set according to the result of judgment (e.g., Loui teaches in 
column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 
teaches an albuming algorithm that arranges the picture according to an event or a sub-event 
wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
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pictures for each event and sub-event Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries, e.g., the distance is either zero or larger than a threshold value so that the photo is 
judged to be either within or outside the event and sub-event boundaries); 

Generating the album data according to the result of classification (e.g., Loui teaches in 
column 7-11 the page layout module presents to the user an automatically laid out album 
organized by event with the algorithm choosing the albuming parameters by default or specified 
by the user). 

However, it remains to be determined whether Loui 5 s query using the query metric meets 
the claim limitation of calculating a distance between the photography location of the image data 
set and a predetermined reference position, based on the location data. It can be seen from Loui 
that the query for the images to be shown in a photo album requires a predetermined event such 
as the predetermined photography location for the query and the images found after query meets 
the criteria set forth in the query metric in which the distance between the queried image and the 
predetermined photography location is within the event boundary. 

This is manifested as follows. Loui teaches in column 9-11 adding annotations for 
pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
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defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries. 

It remains to show whether that Loui implicitly teaches judging whether the distanced 
over a predetermined threshold value. Loui that the query for the images to be shown in a photo 
album requires a predetermined event such as the predetermined photography location for the 
query and the images found after query meets the criteria set forth in the query metric in which 
the distance between the queried image and the predetermined photography location is within the 
event boundary. The event boundary for an event occurs within a region of interest, for example, 
a park or a city requires the calculation of the distance between the locations for the images 
being queried and the query location of interest and the distance should be less than a threshold 
such as the zero threshold for exact query or within a predetermined threshold for approximate 
query in a query language. 
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Nevertheless, Wilcock explicitly discloses using threshold for classifying the pictures 
within a location of interest and therefore Wilcok discloses the claim limitation of judging 
whether the distance is over a predetermined threshold value ( See Wilcok column 7, lines 28- 
51). 

Moreover, Wilcock discloses the claim limitation of calculating a distance between the 
photography location of the image data set and a predetermined reference position, based on the 
location data (column 10, lines 35-65). 

Therefore, taking the combined teachings of Loui and Wilcock, one of the ordinary skill 
in the art would have incorporated Wilcock's thresholding into Loui's method because these 
references deal with the image queries based on the location of interest using a criteria set forth 
in the query wherein Wilcock specifically uses a threshold number to classify the groups of 
pictures. One of the ordinary skill in the art would have been motivated to use the thresholding to 
specifically classify the pictures when they fall into a number of categories of events ( See 
Wilcok column 7, lines 28-5 1 ). 

It remains to show whether that Loui and Wilcok implicitly teach the claim limitation of 
"loading the image data into a personal computer" and "forwarding image data from a user 
terminal of said personal computer to a generation support server, a user at the user terminal 
being identified by a user ID, where a predetermined reference position corresponds to said user 
ID." 

However, Wilcok further discloses the claim limitation of "loading the image data into a 
personal computer" and "forwarding image data from a user terminal of said personal computer 
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to a generation support server, a user at the user terminal being identified by a user ID, where a 
predetermined reference position corresponds to said user ID". 

Wilcok discloses loading the image data into a personal computer (column 10, lines 35- 
65) and forwarding image data from camera/personal digital assistant (See column 10, lines 35- 
65) to PC 5 and/or database 7-9 on the PC or external, a user at the user terminal (comprising of 
the mobile device in combination of the camera and the location server for providing the 
location data for the photos taken; see column 10, lines 55-65) being identified by camera ID 
and a user ID including the location data; see column 3, lines 45-65 and column 10, lines 10-30; 
where a predetermined reference position (the location data of the last taken photo; see column 
9, lines 55-67 and column 10, lines 1-7 and column 10, lines 40-65 wherein the cited reference 
teaches the location data of an individual photo is associated with the last taken photo for 
correlations with the photos being taken with the predetermined reference position can be 
position-in-sequence of photographs) such as the reference location data is provided for 
correlation with the taken photos. 

The reference location/position is provided by the location-based services corresponding 
to the user's location data. The photo location data is related to the camera and/or the personal 
digital assistant's location when a user takes a photo and the user operates a button of the camera 
and/or personal digital assistant to cause the camera or the personal digital assistant to trigger a 
determination of its location; see column 11, lines 10-67 and the location data can be provided 
by a digital camera equipped with GPS wherein a predetermined location reference such as the 
GPS location is associated with the camera or the personal digital assistant identified by the user 
ID. 
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One of the ordinary skill in the art would have been motivated to use the forwarding 
means so that the photos can be stored for retrieval on the basis of their GPS locations and album 
collections are made by matching the location data associated with photo image data in the 
album program ( See Wilcok column 13, lines 20-67 and column 14, lines 1-57 V 

Loui and Wilcok are silent to the claim limitation of displaying together, on a display 
screen, both map data indicating a travel route and thumbnail images in a chronological order. 

However, Asami discloses in Figs. 10-12 and 15-19 displaying together on a display 
screen both map data indicating a travel route and thumbnail images in a chronological order 
(See also Asami column 20, lines 34-62, e.g., the map data indicating the sightseeing course of 
the tour or the map data indicating the route connecting the thumbnail icons of photos). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Loui and Wilcok' s method and apparatus for 
generating an album because Loui and Wilcok' s photos are location dependent in a trip and are 
ordered in chronological order in a trip and Loui and Wilcok' s photos thus can be displayed 
according to the teaching of Asami (Figs. 15-19 and 10-12 of Asami) as Asami's photos are also 
locations dependent and are ordered chronologically for a trip. Wilcok teaches in Figs. 6-7 that 
the thumbnails are displayed on the map having a plurality of routes that a user can travel along. 

One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19, Wilcok Figs. 
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6-7 wherein the thumbnails are displayed on the map having a plurality of routes that a user can 
travel along). 



Re claims 3-5, 8, 10, 13, 15, 18 and 20, Loui teaches in column 8-11 adding annotations 
for pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value for judging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries, e.g., the distance is either zero or 
larger than a threshold value so that the photo is judged to be either within or outside the event 
and sub-event boundaries. Loui further teaches in column 7-11 the page layout module presents 
to the user an automatically laid out album organized by event with the algorithm choosing the 
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albuming parameters by default or specified by the user. Therefore, Loui further discloses the 
claim limitation of "obtaining related data, which is related to the photography location of the at 
least one image data set for which the distance is over the predetermined threshold value, based 
on the location data attached thereto, from a related data storage means which stores a plurality 
of related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 

In response to applicant's argument that "the images are not sent from a computer 
terminal to a server for processing." However, the claim 1 set forth the claim limitation of 
"a user terminal" rather than a computer terminal* In response to applicant's argument 
that "although a user ID is used in Wilcock to organize the images, it is not used to identify 
a predetermined reference position for that particular user." However, the claim 1 recites 
the claim limitation of "a predetermined reference position is associated with said user ID". 
Applicant's arguments are not found in the claim 1 as currently amended. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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Claims 1, 6, 11, 16 and 25 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Kahn et al. US 2004/0004663 Al (hereinafter Kahn) in view of Asami U.S. Patent No. 6,882,350 
(hereinafter Asami). 

Re Claims 1, 6, 11, 16 and 25: 

Kahn teaches a method for generating an album based on album data including at least 
one image data set, which has been photographed during a trip and which has location data 
representing a photography location attached thereto, comprising the steps of: 

Loading the image data into a personal computer (See Paragraph 0044-0045 wherein the 
image data is taken by the digital camera 100 and is loaded to the local host device such as cell 
phone, personal digital assistant, computer or the like and the personal digital assistant or the 
computer meets the claim limitation of "a personal computer")', 

Forwarding image data from a user terminal of said personal computer to a generation 
support server (e.g., Kahn teaches forwarding/uploading user 's photos from the local host device 
to the image management server of Fig. 4; see Paragraph 0089 and 0097-0098), a user at the 
user terminal being identified by a user ID (user 's GPS location, location ID in paragraph 0094 
and 0097 and user 's identity ID of paragraph 0098), where a predetermined reference position 
corresponds to said user ID (e.g., Paragraph 0051 wherein the location information of where the 
user is currently located, the most recently captured images for organizing a plurality of photos. 
The location information may also be a centroid of a given geographic location of interest of 
Kahn meets the claim limitation of (i a predetermined reference position" wherein the centroid is 
associated with the user 's location ID and/or the user 's ID; see paragraph 0093); and, 

The generation support server performing the steps of: 
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Calculating said distance between the photography location of the image data set and a 
predetermined reference position, based on the location data (e.g., Kahn teaches calculating the 
distance by determining whether the user f s GPS location lies within a given geographic location 
of interest to determine the distance between the user *s GPS location and the centroid for a 
given geographic location as performed within an SQL stored procedure; see Paragraph 0093); 

Judging whether the distance is over a predetermined threshold value (e.g., the radius of 
a given geographic location as taught in Kahn meets the claim limitation of "a predetermined 
threshold value "; see Paragraph 0093-0094; a list of fulfillers within a given distance as 
retrieved from the database in the image manager are returned to the local applet and displayed 
to the user as sorted by least distance to the user; see Paragraph 0098-0100); 

Classifying the image data set according to the result of judgment ("albums" based on 
location of Paragraph 0098-0100); and 

Generating the album data according to the result of classification ("albums" based on 
location of Paragraph 0098-0100). 

Kahn is silent to the claim limitation of displaying together, on a display screen, both 
map data indicating a travel route and thumbnail images in a chronological order. 

However, Asami discloses in Figs. 10-12 and 15-19 displaying together on a display 
screen both map data indicating a travel route and thumbnail images in a chronological order 
(See also Asami column 20, lines 34-62, e.g., the map data indicating the sightseeing course of 
the tour or the map data indicating the route connecting the thumbnail icons of photos). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Kahn's method and apparatus for generating an 
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album because Kahn's photos are location dependent in a trip and are ordered in chronological 
order in a trip and Kahn's photos thus can be displayed according to the teaching of Asami (Figs. 
15-19 and 10-12 of Asami) as Asami's photos are also locations dependent and are ordered 
chronologically for a trip. 

One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19). 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1, 3-5, 6, 8, 10, 11, 13, 15, 16, 18, 20 and 21-25 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Loui et al. U.S. Patent No. 6,636,648 (hereinafter Loui), in 
view of Wilcock et al. U.S. Patent No. 6,741,864 (hereinafter Wilcock) and Asami U.S. Patent 
No. 6,882,350 (hereinafter Asami). 

Re claims 1, 6, 11, 16 and 25, Loui discloses a method for generating an album based on 
album data including at least one image data set, which has been photographed using a digital 
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camera during a trip and which has location data representing a photography location attached 
thereto, comprising the steps of: 

Calculating a distance between the photography location of the image data set and a 
predetermined reference position, based on the location data (e.g., Loui teaches in column 9-11 
adding annotations for pictures and associating metadata to the pictures. Loui teaches an 
albuming algorithm that arranges the picture according to an event or a sub-event wherein the 
event or sub-event are defined by the metadata including metadata tags and GPS information to 
determine location and subject depth information. The information defines the context 
surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event. Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries)', 

Judging whether the distance is over a predetermined threshold value (e.g., Loui teaches 
in column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 
teaches an albuming algorithm that arranges the picture according to an event or a sub-event 
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wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
pictures for each event and sub-event Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries); 

Classifying the image data set according to the result of judgment (e.g., Loui teaches in 
column 8-11 adding annotations for pictures and associating metadata to the pictures. Loui 
teaches an albuming algorithm that arranges the picture according to an event or a sub-event 
wherein the event or sub-event are defined by the metadata including metadata tags and GPS 
information to determine location and subject depth information. The information defines the 
context surrounding a picture capturing event allowing the user to select the set of pictures in the 
database using an arbitrary query based on the stored metadata and the selected set of pictures 
is filtered by omitting those pictures whose image quality metric is below some threshold and the 
event segmentation stage determines event and sub-event boundaries to determine final set of 
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pictures for each event and sub-event Loui classified the pictures using the metadata having the 
location information and the pictures are classified in accordance to the predetermined way of 
dealing with the actual boundary condition of events and sub-events. This classification 
algorithm according to the event and sub-event boundary involves a predetermined threshold 
value for judging the distance between the photo location contained in the metadata and a 
reference position associated with an event or sub-event such as the event and sub-event 
boundaries, e.g., the distance is either zero or larger than a threshold value so that the photo is 
judged to be either within or outside the event and sub-event boundaries); 

Generating the album data according to the result of classification (e.g., Loui teaches in 
column 7-11 the page layout module presents to the user an automatically laid out album 
organized by event with the algorithm choosing the albuming parameters by default or specified 
by the user). 

However, it remains to be determined whether Loui's query using the query metric meets 
the claim limitation of calculating a distance between the photography location of the image data 
set and a predetermined reference position, based on the location data. It can be seen from Loui 
that the query for the images to be shown in a photo album requires a predetermined event such 
as the predetermined photography location for the query and the images found after query meets 
the criteria set forth in the query metric in which the distance between the queried image and the 
predetermined photography location is within the event boundary. 

This is manifested as follows. Loui teaches in column 9-11 adding annotations for 
pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
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defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries. 

It remains to show whether that Loui implicitly teaches judging whether the distance is 
over a predetermined threshold value. Loui that the query for the images to be shown in a photo 
album requires a predetermined event such as the predetermined photography location for the 
query and the images found after query meets the criteria set forth in the query metric in which 
the distance between the queried image and the predetermined photography location is within the 
event boundary. The event boundary for an event occurs within a region of interest, for example, 
a park or a city requires the calculation of the distance between the locations for the images 
being queried and the query location of interest and the distance should be less than a threshold 
such as the zero threshold for exact query or within a predetermined threshold for approximate 
query in a query language. 
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Nevertheless, Wilcock explicitly discloses using threshold for classifying the pictures 
within a location of interest and therefore Wilcok discloses the claim limitation of judging 
whether the distance is over a predetermined threshold value ( See Wilcok column 7, lines 28- 
51). 

Moreover, Wilcock discloses the claim limitation of calculating a distance between the 
photography location of the image data set and a predetermined reference position, based on the 
location data (column 10, lines 35-65). 

Therefore, taking the combined teachings of Loui and Wilcock, one of the ordinary skill 
in the art would have incorporated Wilcock' s thresholding into Loui's method because these 
references deal with the image queries based on the location of interest using a criteria set forth 
in the query wherein Wilcock specifically uses a threshold number to classify the groups of 
pictures. One of the ordinary skill in the art would have been motivated to use the thresholding to 
specifically classify the pictures when they fall into a number of categories of events ( See 
Wilcok column 7, lines 28-5 1 ). 

It remains to show whether that Loui and Wilcok implicitly teach the claim limitation of 
"loading the image data into a personal computer" and "forwarding image data from a user 
terminal of said personal computer to a generation support server, a user at the user terminal 
being identified by a user ID, where a predetermined reference position corresponds to said user 
ID." 

However, Wilcok further discloses the claim limitation of "loading the image data into a 
personal computer" and "forwarding image data from a user terminal of said personal computer 



Application/Control Number: 10/669,593 Page 22 

Art Unit: 2628 

to a generation support server, a user at the user terminal being identified by a user ID, where a 
predetermined reference position corresponds to said user ID". 

Wilcok discloses loading the image data into a personal computer (column 10, lines 35- 
65) and forwarding image data from camera/personal digital assistant (See column 10, lines 35- 
65) to PC 5 and/or database 7-9 on the PC or external, a user at the user terminal (comprising of 
the mobile device in combination of the camera and the location server for providing the 
location data for the photos taken; see column 10, lines 55-65) being identified by camera ID 
and a user ID including the location data; see column 3, lines 45-65 and column 10, lines 10-30; 
where a predetermined reference position (the location data of the last taken photo; see column 
9, lines 55-67 and column 10, lines 1-7 and column 10, lines 40-65 wherein the cited reference 
teaches the location data of an individual photo is associated with the last taken photo for 
correlations with the photos being taken with the predetermined reference position can be 
position-in-sequence of photographs) such as the reference location data is provided for 
correlation with the taken photos. 

The reference location/position is provided by the location-based services corresponding 
to the user's location data. The photo location data is related to the camera and/or the personal 
digital assistant's location when a user takes a photo and the user operates a button of the camera 
and/or personal digital assistant to cause the camera or the personal digital assistant to trigger a 
determination of its location; see column 11, lines 10-67 and the location data can be provided 
by a digital camera equipped with GPS wherein a predetermined location reference such as the 
GPS location is associated with the camera or the personal digital assistant identified by the user 
ID. 
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One of the ordinary skill in the art would have been motivated to use the forwarding 
means so that the photos can be stored for retrieval on the basis of their GPS locations and album 
collections are made by matching the location data associated with photo image data in the 
album program (See Wilcok column 13, lines 20-67 and column 14, lines 1-57 ). 

Loui and Wilcok are silent to the claim limitation of displaying together, on a display 
screen, both map data indicating a travel route and thumbnail images in a chronological order. 

However, Asami discloses in Figs. 10-12 and 15-19 displaying together on a display 
screen both map data indicating a travel route and thumbnail images in a chronological order 
(See also Asami column 20, lines 34-62, e.g., the map data indicating the sightseeing course of 
the tour or the map data indicating the route connecting the thumbnail icons of photos). 

It would have been obvious to one of the ordinary skill in the art at the time the invention 
was made to have incorporated Asami into Loui and Wilcok' s method and apparatus for 
generating an album because Loui and Wilcok's photos are location dependent in a trip and are 
ordered in chronological order in a trip and Loui and Wilcok's photos thus can be displayed 
according to the teaching of Asami (Figs. 15-19 and 10-12 of Asami) as Asami's photos are also 
locations dependent and are ordered chronologically for a trip. Wilcok teaches in Figs. 6-7 that 
the thumbnails are displayed on the map having a plurality of routes that a user can travel along. 

One of the ordinary skill in the art would have been motivated to incorporate Asami to 
display the photo icons along the travel route (See also Asami column 20, lines 34-62, e.g., the 
map data indicating the sightseeing course of the tour or the map data indicating the route 
connecting the thumbnail icons of photos; See also Asami Figs. 10-12 and 15-19, Wilcok Figs. 
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6-7 wherein the thumbnails are displayed on the map having a plurality of routes that a user can 
travel along). 

Re claims 3-5, 8, 10, 13, 15, 18 and 20, Loui teaches in column 8-11 adding annotations 
for pictures and associating metadata to the pictures. Loui teaches an albuming algorithm that 
arranges the picture according to an event or a sub-event wherein the event or sub-event are 
defined by the metadata including metadata tags and GPS information to determine location and 
subject depth information. The information defines the context surrounding a picture capturing 
event allowing the user to select the set of pictures in the database using an arbitrary query based 
on the stored metadata and the selected set of pictures is filtered by omitting those pictures 
whose image quality metric is below some threshold and the event segmentation stage 
determines event and sub-event boundaries to determine final set of pictures for each event and 
sub-event. Loui classified the pictures using the metadata having the location information and the 
pictures are classified in accordance to the predetermined way of dealing with the actual 
boundary condition of events and sub-events. This classification algorithm according to the event 
and sub-event boundary involves a predetermined threshold value forjudging the distance 
between the photo location contained in the metadata and a reference position associated with an 
event or sub-event such as the event and sub-event boundaries, e.g., the distance is either zero or 
larger than a threshold value so that the photo is judged to be either within or outside the event 
and sub-event boundaries. Loui further teaches in column 7-11 the page layout module presents 
to the user an automatically laid out album organized by event with the algorithm choosing the 
albuming parameters by default or specified by the user. Therefore, Loui further discloses the 
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claim limitation of "obtaining related data, which is related to the photography location of the at 
least one image data set for which the distance is over the predetermined threshold value, based 
on the location data attached thereto, from a related data storage means which stores a plurality 
of related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 

Re Claims 21-24: Loui and Wilcock disclose the query language for the images to be 
classified in a album including the location of interest supplied in a query and therefore, the cited 
reference disclose the predetermined reference position registered as user data in a query. 

Claims 2, 7, 9, 12, 14, 17, and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Loui et al. U.S. Patent No. 6,636,648 (hereinafter Loui), in view of Wilcock et 
al. U.S. Patent No. 6,741,864 (hereinafter Wilcock) and Asami U.S. Patent No. 6,882,350 
(hereinafter Asami), in view of Kino et al. U.S. Patent No. 6,832,101 (hereinafter Kino). 

Re claims 2, 7, 12, and 17, Loui teaches in column 8-11 adding annotations for pictures 
and associating metadata to the pictures. Loui teaches an albuming algorithm that arranges the 
picture according to an event or a sub-event wherein the event or sub-event are defined by the 
metadata including metadata tags and GPS information to determine location and subject depth 
information. The information defines the context surrounding a picture capturing event allowing 
the user to select the set of pictures in the database using an arbitrary query based on the stored 
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metadata and the selected set of pictures is filtered by omitting those pictures whose image 
quality metric is below some threshold and the event segmentation stage determines event and 
sub-event boundaries to determine final set of pictures for each event and sub-event. Loui 
classified the pictures using the metadata having the location information and the pictures are 
classified in accordance to the predetermined way of dealing with the actual boundary condition 
of events and sub-events. This classification algorithm according to the event and sub-event 
boundary involves a predetermined threshold value forjudging the distance between the photo 
location contained in the metadata and a reference position associated with an event or sub-event 
such as the event and sub-event boundaries, e.g., the distance is either zero or larger than a 
non-zero threshold value so that the photo is judged to be either within or outside the event and 
sub-event boundaries. Loui further teaches in column 7-11 the page layout module presents to 
the user an automatically laid out album organized by event chronologically or non- 
chronologically with the algorithm choosing the albuming parameters by default or specified by 
the user. The laid out album organized by events arranged in a chronological order requires the 
location information for the events and sub-events along the route. Therefore, Loui further 
discloses the claim limitation of "generating travel route data, which represents a route taken 
during the trip, based on the location data attached to the at least one image data set for which the 
distance is over the predetermined threshold value" because classifying the events and sub- 
events based on the location are travel route data which represents a route taken during a trip for 
locations of the pictures arranged in a chronological order. Loui further discloses generating 
photography data such as the picture data and the metadata which represents that the image data 
set was obtained at the photography locations along the route. 
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Loui does not disclose "obtaining a map data set that contains the route from a map 
database which stores a plurality of map data sets, based on the travel route data" and similarly 
"attaching the photography data to the map data set and generating album data including the map 
data set, in which the photography data is correlated with the image data set." 

However, Kino discloses a method of photographing a photo image at a construction side 
to be related with a map and registering it in the database wherein the information about the 
photograph position on the map is stored in the database to be related with attached information 
about the construction site. Kino further discloses associating a symbol to the place where the 
image was photographed on the map allowing for searching the image and the attached 
information to be related each other by referring to the symbol list using the symbol ID. Kino 
thus discloses the locations of the travel route data chronologically because the locations of the 
pictures taken are ordered chronologically along a travel route. In this way, the locations are 
searched/extracted from the database based on the attached information and the locations thus 
extracted in a chronological order form a travel route (See Kino column 3, lines 50 through 
column 4, lines 16). 

Although Loui did not specifically teach the claim limitation, it is well known to one of 
the ordinary skill in the art to have incorporated Kino's map database to have associated symbols 
or metadata's locations to the photographs and to have determined a travel route based on the 
Loui's location data associated with the pictures. 

One having the ordinary skill in the art would have been motivated to do this because it 
would have provided a travel route on the map database and means for relating the position on 
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the map with the symbols attached to or associated with the pictures (See Kino column 3, lines 
50 through column 4, lines 16). 

Re claims 9, 14 and 19, Loui teaches in column 8-11 adding annotations for pictures and 
associating metadata to the pictures. Loui teaches an albuming algorithm that arranges the 
picture according to an event or a sub-event wherein the event or sub-event are defined by the 
metadata including metadata tags and GPS information to determine location and subject depth 
information. The information defines the context surrounding a picture capturing event allowing 
the user to select the set of pictures in the database using an arbitrary query based on the stored 
metadata and the selected set of pictures is filtered by omitting those pictures whose image 
quality metric is below some threshold and the event segmentation stage determines event and 
sub-event boundaries to determine final set of pictures for each event and sub-event. Loui 
classified the pictures using the metadata having the location information and the pictures are 
classified in accordance to the predetermined way of dealing with the actual boundary condition 
of events and sub-events. This classification algorithm according to the event and sub-event 
boundary involves a predetermined threshold value forjudging the distance between the photo 
location contained in the metadata and a reference position associated with an event or sub-event 
such as the event and sub-event boundaries, e.g., the distance is either zero or larger than a 
threshold value so that the photo is judged to be either within or outside the event and sub-event 
boundaries. Loui further teaches in column 7-11 the page layout module presents to the user an 
automatically laid out album organized by event with the algorithm choosing the albuming 
parameters by default or specified by the user. Therefore, Loui further discloses the claim 
limitation of "obtaining related data, which is related to the photography location of the at least 
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one image data set for which the distance is over the predetermined threshold value, based on the 
location data attached thereto, from a related data storage means which stores a plurality of 
related data sets" wherein the related data set is the metadata including the location of 
photographing stored in the database. Loui further discloses the claim limitation of "generating 
album data, which includes the related data" because Loui discloses displaying album organized 
by event wherein the event are associated with the pictures in the album. 



Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (571) 272-7665. 
The examiner can normally be reached on 8:00 - 6:30 (Mon-Thu). 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on (571) 272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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